Animal-Wearable First Person View System

ABSTRACT

An animal-wearable head-mountable display (AW-HMD) device comprises a head fitting designed to fit the head of an animal, and an output subsystem coupled to or integral with the head fitting and configured to output a signal to the animal. The device can comprise an optical module configured to project images into an eye of the animal, and various other modules, such as an audio module, a tactile module and/or an olfactory module. An animal-wearable first person view (AW-FPV) system may be used separately or in conjunction with the AW-HMD device, to enable a remote human user to view the animal&#39;s real-world environment from the animal&#39;s point of view.

This application is a continuation-in-part of U.S. patent applicationSer. No. 16/030,672, filed on Jul. 9, 2018, which claims the benefitU.S. provisional patent application No. 62/537,883, filed on Jul. 27,2017, each of which is incorporated by reference herein in its entirety.This application further claims the benefit of U.S. provisional patentapplication No. 62/581,501, filed on Nov. 3, 2017, which is incorporatedby reference herein in its entirety.

FIELD

At least one embodiment of the present disclosure pertains tohead-mountable display and viewing devices, and more particularly, to ahead-mountable display device designed to be worn by an animal andhaving a first person viewing system.

BACKGROUND

Head mountable display (HMD) devices are available on the market today,but they are designed only for use by humans. Thus, their current formfactors, optical arrays and software are not suitable for use withanimals. Additionally, certain camera products are available that can beput on an animal and used to acquire images approximately from theanimal's perspective, such as GoPro cameras, which can be put on a dog'sback. However, these products are also generally intended for human useand are not optimal for use with animals, due to problems with imagestability and other factors.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the present disclosure are illustrated by wayof example and not limitation in the figures of the accompanyingdrawings, in which like references indicate similar elements.

FIG. 1 illustrates an overview of an Animal Wearable Head MountableDisplay (AW-HMD) system.

FIG. 2A is an example of a side cross-sectional view of a side-mountedoptical module that can be used for this purpose in the AW-HMD device.

FIG. 2B schematically illustrates the operation of the optical module ofFIG. 2A.

FIG. 3A shows a front/side perspective view of an embodiment of theAW-HMD device mounted on the head of an animal.

FIG. 3B shows a front view of the embodiment of AW-HMD device shown inFIG. 3A.

FIG. 4A shows a side view of an embodiment in which the optical moduleis integrated with goggles.

FIG. 4B shows a front/side perspective view of an embodiment in whichthe optical module is integrated with goggles.

FIG. 4C shows a front view of an embodiment in which the optical moduleis integrated with goggles.

FIG. 4D shows an animal wearing the AW-HMD device, including headset andvest portions.

FIG. 5 shows an embodiment of AW-HMD device 1 with olfactory and tactileenhancement and detection capability.

FIG. 6 shows a canine eye to illustrate certain principles ofeye-tracking.

FIG. 7 depicts an example of a scenario that may be encountered duringuse of the AW-HMD device.

FIG. 8 depicts another example of a scenario that may be encounteredduring use of the AW-HMD device.

FIG. 9 is a block diagram of the major components of the AW-HMD deviceaccording to at least one embodiment.

FIG. 10 shows a front and bottom perspective view of an animal-wearablefirst person view (AW-FPV) device.

FIG. 11 shows how the AW-FPV device can mount to an AW-HMD.

FIG. 12 shows a side and partial front perspective view of the AW-FPVdevice.

FIG. 13 shows an example of the internal components of the AW-FPVdevice.

FIG. 14 shows an example of the mounting and locking mechanisms of theAW-FPV device.

FIG. 15 shows an example of mounting of the AW-FPV device tocanine-adapted goggles.

FIG. 16 shows an example of a user interface that may be used to viewimages captured by the AW-FPV device and/or to control functions of theAW-FPV device.

DETAILED DESCRIPTION

In this description, references to “an embodiment”, “one embodiment” orthe like, mean that the particular feature, function, structure orcharacteristic being described is included in at least one embodiment ofthe technique introduced here. Occurrences of such phrases in thisspecification do not necessarily all refer to the same embodiment. Onthe other hand, the embodiments referred to also are not necessarilymutually exclusive.

Introduced herein are a device and system for communication between ananimal and a human, that enables direct, specific communication ofinstructions to the animal, such as regarding desired direction and/ordesire of the human. Note that the term “animal” as used herein isintended to exclude humans. The system can include an Animal WearableHead Mountable Display (AW-HMD) device that provides visual cuesdirectly to an animal at the direction of the handler, who may belocated remotely from the animal (i.e., not within the animal's normalrange of hearing or vision). The system also or alternatively include anAnimal Wearable First Person View (AW-FPV) system that enables directviewing by a human of the animal's visual perspective. Further, thesystem may provide nonvisual signals to the animal, such as tactile(e.g., haptic), auditory or olfactory signals, or any combinationthereof, which may be based on remote user inputs from the human.

I. Animal Wearable Head-Mounted Display System

At least some of the embodiments of the AW-HMD device introduced herecomprise a headset portion that includes a head fitting in the form of aruggedized goggle designed to accommodate the distinct form factor of ananimal's head (generally though not necessarily designed to optimallyfit a particular animal species), a light source, a lens array, cameraoptics, and controls and feedback mechanisms for auditory, olfactoryand/or tactile data. The AW-HMD enables viewing by a human of ananimal's direct line of sight and visual communication of images backinto the line of sight of the animal. The device can be used to providesignals to the animal so that there is no ambiguity in direction,target, or any other visual reference point. The AW-HMD device caninclude non-visual sensors, multiple cameras and integrated opticalequipment and associative computing and processing power. Thesecomponents may include optics equipment (e.g., display, one or morelenses, beamsplitter) to display visual content to the animal, anoutward facing camera to capture the animal's line of sight, and aninward facing camera to address alignment between the animal's eye andoptics and/or to perform eye tracking. Additional sensory and spectraldetection sensors can also be included, along with custom-designedsoftware to address issues related to mammal-specific visual acuityand/or other species-specific visual acuity.

The AW-HMD device disclosed herein is designed for animals, with atleast some contemplated embodiments being particularly designed formammals, and more particularly, for canines. However, the principlesintroduced here can additionally or alternatively be applied and/oradapted for use with other species, including non-mammalian species. Atleast some embodiments of the system introduced here have the followingcapabilities: (1) permit delivery of information into the line of sightand/or perception of the animal without discernment of such informationby third parties; (2) provide direct line of sight (“first-person”viewing) of what the animal sees (here the term “person” in“first-person” does not refer to a human); (3) integrate olfactoryresponses into the design; (4) adjust for varied lighting scenarios; (5)automatically calibrate to the eye position of the animal; (6) integratehaptic controls for additional communications; and (7) integrate audiocontrols for bi-directional audio communication.

At least some embodiments of the AW-HMD system may also have one or moreof the following: a near field-light source, an optical array, aflexible near-to-eye display (NED), an infrared camera and imaging todetect heat sources, optical arrays for use in veterinary/clinicalpractices to test animal visual acuity and perception, spectral sensorsto detect external anomalies, audio components, pupil tracking, and/orhaptics.

FIG. 1 shows an overview of the AW-HMD system. The AW-HMD system 10 inthe illustrated embodiment includes an AW-HMD device 1 that includes aheadset portion comprising goggles 2 to be worn on the head of theanimal, such that the goggles 2 cover one or both eyes of the animal.The AW-HMD device 1 can perceive the outside world and is aware ofobjects and conditions in the environment through sensors such as rangefinders, IR sensors, camera(s), or the like. The AW-HMD device 1 maycommunicate through a networked connection 4 (e.g., via one or morewired and/or wireless networks) with remote user interfaces and device3. The remote user interfaces and device 3 may provide a user interfaceto allow a human user to control functions of the AW-HMD device 1through a networked connection 4. Through the networked connection 4 andan audio module 5 in the AW-HMD device 1, a user can use the remote userinterfaces and device 3 to communicate bi-directionally with the AW-HMDdevice 1 and to communicate with the animal.

The AW-HMD device 1 may have any of various different form factors tofit any of various different functional and anthropomorphicrequirements. In some embodiments, the AW-HMD device 1 may be designedwith conventional goggles 2, as noted above. The goggles 2 may at leastpartially enclose active computer graphics displays, which may beconfigured as see-through (at least partially transparent) displays,such that digital imagery and objects can be overlaid on the animal'sview of its environment. Any of various see-through display technologiesand optical designs may be used, such as emissive displays (e.g., lightemitting diode (LED), organic LED (OLED), or active-matrix OLED(AMOLED)), holographic displays, or the like. The optical configurationmay include a single reflexive display, monocular display, or binoculardisplay.

The AW-HMD device 1 may also have a number of integrated computingelements, which may include one or more integrated microprocessorsand/or digital signal processors (DSPs), power management, as well aswired and wireless communications transceivers (e.g., for universalserial bus (USB), cellular, WiFi, Bluetooth, mesh connections, etc.).The AW-HMD device 1 may also have one or more positional sensors, suchas global positioning system (GPS) circuitry, accelerometers, inertialmeasurement unit (IMU), or the like. It may also have other sensors,such as a camera(s), rangefinders, microphones, speakers, hyper-spectralcamera, spectral illuminators, temperature sensors, olfactory detection,Geiger counter, and the like.

Some embodiments of the AW-HMD device 1 may include eye-tracking sensorsto determine and assist in the alignment of the optical display(s) tothe animal's eye(s). Data gathered by eye-tracking sensors may also bemade available through a networked connection 4 to remote userinterfaces and devices 3, such that the external interfaces and devices3 can be alerted to changes in the eye-box relative to the animal'sactive field of view (FOV). Additionally, embodiments of the AW-HMDdevice 1 may also optimize the delivery of imagery to the animal throughthe display such that colors are optimized for the animal or animalspecies (e.g., dichromacy for canines) as well as for optimal contrast.Additionally, the AW-HMD device 1 may generate images and objects fordisplay to the animal by using shapes optimized for the visualperception capabilities of a particular animal or animal species, whichshapes may include linear, oblong, elliptical, orbicular, lanceolate,ovate, oblanceolate, obvate, triangular and deltate shape types.

The AW-HMD device 1 may also have integrated command and controlfeatures, which may include contextually based control, active control,user control, remote control, passive control, or the like. For example,the AW-HMD device 1 may have an integrated sensor, such as a camera andassociated detection circuitry that can perform object recognition(e.g., facial, landmark, or the like) on captured images, such that theintegrated processing system can interpret an object and relay anindication of the identified object to the remote user interfaces anddevice 3 through a networked connection 4. The AW-HMD device 1 may alsoprovide various types of signals to the animal directly (automaticallyand/or based on user inputs) through the optical display.

The AW-HMD device 1 may also adjust itself based on measured orperceived environmental conditions, such as ambient light. For example,in dark environments the AW-HMD device 1 may decrease the level ofcontrast of a digitally displayed object or image. In some embodiments,the control technology may be mounted on the AW-HMD device 1 such that auser can make manual adjustments directly to the AW-HMD device 1.Alternatively or additionally, some embodiments may include the abilityto make adjustments to the contrast of displayed objects through anremote user interface and device 3 through a networked connection 4.

Additionally, some embodiments of the AW-HMD device 1 may include activesensory feedback. For example, embodiments of the AW-HMD device 1 mayhave integrated olfactory delivery capability, such that a userconnected through a networked remote user interface 3 may triggerdelivery of olfactory cues through an olfaction sensor and deliverycomponent 7 on the AW-HMD device 1, to provide signals to the animal.Some embodiments of AW-HMD device 1 may also include haptic feedbackelements 8 to provide signals to the animal. For example, a user,through a network connection 4 and an remote user interface or device 3,may input into the remote user interface 3 to activate the hapticfeedback elements 8 on AW-HMD device

Some embodiments of the AW-HMD device 1 may have sensors that detectmovement of the animal (e.g., pace change, position, and the like)including accelerometers, gyros, and other internal measurements, wherethe integrated processors may interpret the tracked movement and providequantitative and qualitative measurements of the animal's positionand/or motion to the remote user interfaces and device 3.

As noted above, the AW-HMD device 1 may be in communication with remoteuser interfaces 3. The remote user interfaces 3 may be generated bydevices having any of various different forms. For example, a cell phonescreen may be adapted to receive input from AW-HMD device 1 and may beset up to actively control functional aspects of the AW-HMD device 1.The remote user interface 3 may additionally or alternatively includeother forms, such as one or more tablets, laptop computers, desktopcomputers, or the like. In each case, the remote user interface may alsoinclude sensors (e.g., IMU, accelerometers, compass, temperature, andthe like) to provide additional input in controlling the AW-HMD device 1and instructing the animal.

As mentioned above, the AW-HMD device 1 may include one or more opticalmodules to display images to the animal. FIG. 2A is an example of a sidecross-sectional view of a side-mounted optical module that can be usedfor this purpose in the AW-HMD device 1. The view in FIG. 2A is from theperspective of the animal when the AW-HMD device 1 is being worn by theanimal. The illustrated optical module 201 includes a computer-operateddisplay 202 (such as, for example, an eMagin SXGA096-CFXL OLED display),a beamsplitter/combiner 203 with reflective coatings, a lens array 204that includes a plurality of lenses and/or waveguides 207, and camera205. Note that while the illustrated embodiments shows two lenses 207,other embodiments may have a different number of lenses, which could bejust a single lens. The display 202 emits substantially uniform lightthat is generally directed towards the reflective beamsplitter/combiner203, through the lens array 204. The camera 205 is aligned with thedisplay 202 in the direction in which most light is emitted from thedisplay 202.

A portion of the light that is emitted by the display 202 is reflectedby the reflective beamsplitter/combiner 203 (perpendicularly to theplane of the drawing) into the eye of the animal (not shown). Anotherportion of that emitted light is passed through thebeamsplitter/combiner 203 and captured by the camera 205, such that anyimages produced by the display 202 can be monitored and tracked inthree-dimensional space by remote user interfaces and devices 3.Additionally, a portion of the light that impinges on thebeamsplitter/combiner 203 from the animal's environment is reflected bythe beamsplitter/combiner 203 to the camera 205, which provides imagesof the animal's optical perspective for transmission to remote userinterfaces and devices 3.

The optical module 201 can be lightweight and compact such that it fitsinto a small portion of an AW-HMD device 1. This allows integration ofthe optical module 201 into off-the-shelf canine goggles, such as caninegoggles made by Rex Specs. An example of the manner of integration isillustrated in FIGS. 4A 4B and 4C, discussed further below. Note thatother embodiments of the optical module 201 may instead have top-mountedor bottom-mounted configurations, with correspondingly modifiedconfigurations of the lens array 204 and beamsplitter/combiner 203.

The manner of operation of the optical module of the AW-HMD 1 is furtherexplained now with reference to FIG. 2B. Note that the illustratedfeatures are not necessarily drawn to scale. In FIG. 2B, optical module201 is viewed from a position either directly above or directly belowthe animal's head.

As mentioned above, in at least some embodiments displays digital imagecontent to an eye 239 of an animal wearing the AW-HMD 1 and allows theanimal to see through the display, such that the digital content isperceived by the animal as augmenting the animal's view of thesurrounding environment, i.e., as augmented reality (AR) and/or mixedreality (MR) images. The optical configuration may have a variablytransmissive optical element that is in-line with the animal's view ofits surroundings, such that the degree of transmission of thesee-through view can be increased and decreased. The variablytransmissive optical element may be or include, for example, thebeamsplitter/combiner 203 and/or one or more of the lenses 207. Thisfeature may be helpful in situations where the animal would be betterserved with a high degree of transmission of see-through view and when,in the same AW-HMD device 1, the animal would be better served with alower degree of transmission of see-through transmission. The lowerdegree of see-through transmission may be used, for example, in brightconditions and/or in conditions where higher contrast for the digitallypresented objects are desirable.

The camera captures images (video and/or stills) of the surroundingenvironment by receiving reflected light from the surroundingenvironment off of the beamsplitter/combiner 203, which is in-line withthe animal's see-through view of the surrounding. In some embodiments,the light emission surface of the display element 202 is enclosed by alight shield 402 (FIG. 4B), such that ambient light does not interferewith light reflected 237 and/or transmitted 233 in the direction of thecamera 1101, which is captured by the camera 1101 to enable a remoteuser to view an image of where the object being displayed is being shownto the animal in three-dimensional space.

In some embodiments, as shown, the camera 205 aperture is perpendicularto the direction in which the animal is looking and directly alignedwith the display element 202, as shown in FIGS. 2A and 2B. Image light233 is emitted by the display element 202, and a portion 234 of thatimage light 233 is reflected toward the animal's eye 239 by the inwardfacing surface of the partially reflective beamsplitter/combiner 203.Most of the remaining portion 237 of the image light 233 from thedisplay element 202 simply passes through the beamsplitter/combiner 203to the camera 205 to capture the animal's view of the computer-generatedimages. Additionally, a portion of the light received from the animal'ssurrounding environment (not shown) is reflected off the outward facingsurface of the beamsplitter/combiner 203 and into the camera 205, tocapture the animal's view of its environment, while another portion ofthat light energy from the environment is passed through thebeamsplitter/combiner 203 to the animal's eye 239.

In some embodiments, the beamsplitter/combiner 203 includes a coating onits surface that faces the camera 205, such that visible wavelengthlight is substantially transmitted while infrared light is substantiallyreflected; and the camera 205 captures images that include at least aportion of the infrared wavelength light. In such embodiments, the imagelight 233 includes visible wavelength light, and portion 237 of thevisible wavelength light is transmitted by the beamsplitter/combiner203. This may be useful to remote users through a networked connection 4on remote interfaces and devices 3 to view the image presented by AW-HMDdevice 1 to the animal when in low-light conditions, for example.

As noted, the optical module 201 may contain a lens 207 or an array 204of lenses 207, where light from the display element 202 is projectedthrough the lens or lens array 204 onto the beamsplitter/combiner 203 tooverlay objects onto the animal's view of the real world. Light controlstructures (not shown) can also be included to control the distributionof the light that is delivered by the display element 202. The lightcontrol structures can include, for example, diffusers, ellipticaldiffusers, prism films and lenticular lens arrays, prism arrays,cylindrical lenses, Fresnel lenses, refractive lenses, diffractivelenses or other structures that control the angular distribution of theimage light 233.

Additionally, the optical module 201 is not limited to a side-mounteddisplay nor to a monocular display. Other embodiments, for example, canbe in a binocular display, whereby images and objects are displayed toboth eyes simultaneously. Additionally, other embodiments may include asingle, flexible reflexive surface, where images and objects aredisplayed directly onto that reflexive surface.

An embodiment of the optical system discussed with regard to FIGS. 2Aand 2B can be seen in FIGS. 3A and 3B, showing an example of how theoptical module 201 may be worn on the head 301 of a canine. The actualmounting mechanism, including goggles, is not shown in FIGS. 3A and 3Bfor clarity, but is shown in FIGS. 4A, 4B and 4C. The display element202, beamsplitter/combiner 203, lens array 204 and camera 205 arecoupled together by a pair of brackets 310. The lens array 204 (notvisible in FIGS. 3A and 3B) may be covered by a light shield 402 toreduce interference from ambient light. A flexible cord 403 is used toprovide power from a separate power supply (not shown) to the AW-HMDdevice 1 and to bi-directionally communicate data and control signalsbetween the AW-HMD device 1 and a microprocessor and/or wirelesstransceiver (not shown). As discussed below, the power supply,microprocessor, memory and wireless transceiver may be mounted on or ina vest worn by the animal.

FIGS. 4A, 4B and 4C show three different views illustrating how theoptical module 201 can be integrated with off-the-shelf canine goggles400, such as those made by Rex Specs. Specifically, FIG. 4A shows aleft-side view, FIG. 4B shows a top/left-side view, and FIG. 4C depictsa front view of such an embodiment. A region 408 can be cut out of thelens 407 of the goggles 400, through which the optical module 201 can beinserted and suspended so that the optical module 201 partiallyprotrudes from the goggles 400, as shown. As shown, three mounts 401 canbe used to secure the optical module 201 to the frame of the goggles 401and suspend the optical module 201 in place. As shown, the mounts 401,which may include screws or other suitable fasteners, may be placedabove the goggles lens, below the goggles lens, and to the side of theoptical array 201, for example. Each of the mounts 401 attaches to oneof the brackets 310 within the interior space of the goggles 400.Moreover, some embodiments may include adjustable mounting arms 302,such that appropriate adjustments can be made to align the opticalmodule to the animal's eye. Moreover, to integrate the optical array 201into the pre-existing goggles 400, modification to the front lens of thepre-existing goggles can be done, such that the optical array 201partially protrudes outside of the finished goggle 400, while at thesame time minimizing the opening.

As shown in FIG. 4D, some of the components of the system may be mountedon or in a vest 421 worn by the animal 422, such as the power supply(e.g., battery), microprocessor, memory and wireless transceiver. Thesecomponents may be contained within one or more physical modules 423 thatare attached to the vest and connected to the headset portion 424 of theAW-HMD 1 by one or more cables 403 for providing power to the headsetportion 424 and communicating data between the headset portion 424 andthe components in the module(s) 423. In other embodiments, the one ormore modules may communicate with and/or provide power to the headsetportion 424 wirelessly.

Another consideration relating to integration of an optical module intoan animal-worn form factor is the departure from traditional designprinciples that are based on a more-planar human face, to a designsuitable for an animal with a less-planar face, such as a dog or otheranimal with a prominent snout. To account for these variants, thelocation of the optical array may need to be adjusted so that it canclear, in this present example, the snout of a canine. To enable thisadjustment and to account for potential variants of an animal's facialfeatures while also allowing for the best “eye-box” to be presented tothe animal, in certain embodiments the mechanism used to mount theoptical array 201 to the goggles 400 can allow for rotational adjustmentof the optical array 201 relative to the goggles 400. With this rotationof the optical module 201, it is possible to adjust its positioning forprotruding features, such as the animal's snout. To further account forsuch facial variations, a small portion 206 of the beamsplitter/combiner203 can be cut off near an edge, as shown in FIG. 2A, such that anoptical fit can be made, while maintaining the largest possible eye-boxfor the animal.

FIG. 5 shows an embodiment of AW-HMD device 1 with olfactory enhancementand detection capability (to facilitate illustration, the optical moduleis not shown in this view, but would be present in an actual device).The olfactory sense and response elements 501 enable a remote human userto deliver olfactory cues to the animal and to receive olfactory orolfactory-based cues from the AW-HMD device 1, each via a networkconnection 4 and remote user interfaces and devices 3. Elements 501 havea wired or wireless (e.g., Bluetooth) connection (not shown) with amicroprocessor and/or long-range wireless transceiver (not shown) on theAW-HMD device 1. These elements enable a human handler to detect scents,passively, or actively, in the environment by allowing air to naturallybe inducted or through vacuuming air into the scent discriminationsystem in the AW-HMD device 1. Additionally or alternatively, theolfaction system may deliver scent cues to the animal to direct theanimal or otherwise elicit a scent-based response. In some instances theAW-HMD device 1 and its associative computing elements may be able todeliver scent cues to the animal automatically (i.e., not in response toa direct human input), having previously been programmed by a remoteuser with the appropriate scent detection profile and response.

FIG. 5 also shows an embodiment of the AW-HMD device 1 with tactile(e.g., haptic) sensing and feedback capability. The hapticsensors/feedback elements 504 may enable a remote user through anetworked connection 4 on an remote user interface or device 3 todeliver haptic feedback to the animal, e.g., to control the animal'sdirection of movement or gaze. Additionally, through the use ofassociative computing power, accelerometers and like in the AW-HMDdevice 1, the AW-HMD device 1 may also be able to deliver hapticfeedback to a remote user through a networked connection 4, based onpredetermined inertial measurements that may be recorded via the AW-HMDdevice 1. Haptic feedback, for the AW-HMD device 1 or remote user mayinclude the application of force, vibrations and/or other subtlemovements. Elements 504 can be assumed to have a wired or wirelessconnection with an on-board microprocessor and/or wireless transceiveron the AW-HMD device 1.

FIG. 6 shows a canine eye to illustrate how eye-tracking can be done inaccordance with the technology introduced here. An embodiment of theAW-HMD device 1 with eye-tracking capability may incorporate at leasttwo cameras, including an eye camera to capture an image of the animal'seye and a second camera to capture images of the animal's environment.The figure depicts the pupil of the eye 601 with a pupil tracking area602, and a spot search area 603 for tracking the relation of the pupilwithin the eye-box, such that if the pupil 601 changes in relation tothe eye-box, the corresponding FOV may be relayed to remote userinterfaces and devices 3.

FIG. 7 depicts a scenario that may be encountered during use of theAW-HMD device 1. In this embodiment, the animal wearing the AW-HMDdevice 1 is viewing a building 701. The FOV of the animal that can beaddressed by the AW-HMD device 1, in particular by the optical module201, is represented by dotted box 702. Through a remote user interfaceand device 3, a remote user 704 can view (via a networked connection 4)the FOV addressable by the optical module 201, as represented by thedotted box 702. In addition, through the camera sensor on AW-HMD device1, the remote user 704 may also view beyond the optical module and theanimal's immediate FOV. Additionally, the remote user 704 may input intothe optical module 201 (e.g., through speech or some physical action onthe remote user interface and device 3) to highlight areas in theaddressable FOV 702 of the optic that can be perceived by the animal,which in this instance is shown as a dot 705.

Moreover, the AW-HMD device 1 is not limited to being able to highlightobjects in the physical world that are in the immediate and direct FOVof the optical module 201. The technology introduced here can alsoenable a remote user to tag or otherwise direct their input to subjectmatter outside of the animal's current FOV through the optical module201 but within the FOV of a camera of the AW-HMD device 1. For example,the remote user can “tag” an area or object within the addressable FOVof the camera on the user interface 3 but outside the animal's FOVthrough the optical module 201, which may cause the AW-HMD device 1 todirect the animal (e.g., through visual, haptic or audible signal) toadjust its head position, to bring the tagged area/object within theanimal's FOV through the optical module 201.

FIG. 8 depicts another use scenario, in which the animal wearing theAW-HMD device 1 encounters a group of people 901, and the AW-HMD device1, through object and facial recognition, overlays a cue (e.g., a dot)802 on a user-designated object (e.g., a specific person), even thoughthe object is not within the FOV 803 of the optical module 201. This mayoccur when the camera in the AW-HMD device 1, having a larger FOV 804than the FOV 803 of the optical module 201, detects a desirable objectand tags the object such that the optical module 201 can activate assoon as the object is in the animal's FOV. An example in which this maybe desirable is where a law enforcement unit sends a dog wearing theAW-HMD device 1 into a hostile environment for surveillance. In thiscase, the law enforcement officers may be unaware of hostiles in thearea being surveilled. However, because of the AW-HMD device 1 and itsassociative computing capabilities, if the dog were to come into contactwith a hostile group and a member of that group were to be identified asa “wanted” target, the system may automatically select that “wanted”individual as the desired target to apprehend, as shown by the cue 802.By doing so, the AW-HMD device 1 may allow for trained animals to havehigher mission capabilities, since they are no longer as reliant on thehuman handler for cues. Additionally, embodiments may also allow aremote user to view the objects being tagged through a networkedconnection 4, as shown.

FIG. 9 is a high-level block diagram showing the major components of theAW-HMD device 1 according to at least one embodiment. Note that otherembodiments of the AW-HMD device 1 may not include all of the componentsshown in FIG. 9, and/or may include additional components not shown inFIG. 9.

In the illustrated embodiment, the physical components of the AW-HMDdevice 1 include one or more of each of: a processor 901, a memory 902,an optical module 903, an eye-tracking video camera 904, a video camera905 for imaging the animal's environment in the animal's line of sight,a communication subsystem 906, an audio subsystem 907, a tactilesubsystem 908, and an olfactory subsystem 909, all coupled together(directly or indirectly) by an interconnect 910. Note that in someembodiments, one or more of these components may be located off theheadset portion of the AW-HMD device 1, such as on a vest worn by theanimal, as shown and described with reference to FIG. 4D.

The interconnect 910 may be or include one or more conductive traces,buses, point-to-point connections, controllers, adapters, wireless linksand/or other conventional connection devices and/or media, at least someof which may operate independently of each other.

The processor(s) 901 individually and/or collectively control theoverall operation of the AW-HMD device 1 and perform various dataprocessing and control functions. For example, the processor(s) 901 mayprovide at least some of the computation and data processingfunctionality for generating and displaying computer-generated images tothe animal and/or for providing other signals to the animal (e.g.,auditory, olfactory or haptic). Each processor 901 can be or include,for example, one or more general-purpose programmable microprocessors,digital signal processors (DSPs), mobile application processors,microcontrollers, application specific integrated circuits (ASICs),programmable gate arrays (PGAs), or the like, or a combination of suchdevices.

Data and instructions (code) 911 that configure the processor(s) 901 toexecute aspects of the mixed-reality visualization technique introducedhere can be stored in the one or more memories 902. Each memory 902 canbe or include one or more physical storage devices, which may be in theform of random access memory (RAM), read-only memory (ROM) (which may beerasable and programmable), flash memory, miniature hard disk drive, orother suitable type of storage device, or a combination of such devices.

The optical module 903 may include one or more active display elements(e.g., an OLED display) and associated optics, for displayingcomputer-generated images to the animal. The communication subsystem 906enables the AW-HMD device 1 to receive data and/or commands from, andsend data and/or commands to, a remote processing system, such as remoteuser interfaces and devices 3. The communication subsystem 906 can be orinclude one or more of, for example, a Wi-Fi transceiver, cellulartransceiver (e.g., LTE/4G or 5G), Bluetooth or Bluetooth Low Energy(BLE) transceiver, baseband processor, a universal serial bus (USB)adapter, Ethernet adapter, cable modem, DSL modem, or the like, or acombination thereof.

The audio subsystem 907 can be or include one or more speakers and/orone or more microphones. The tactile subsystem 908 may be or include oneor more haptic actuators to provide haptic signals to the animal, and/orone or more haptic sensors by which to provide haptic feedback to aremote human user. The olfactory subsystem 909 may be or include one ormore olfactory delivery elements to provide olfactory signals to theanimal, and/or one or more olfactory sensors by which to provideolfactory feedback to a remote human user.

II. Animal-Wearable First Person View System

As noted above, some embodiments of the technology introduced hereenable direct viewing by a human of the animal's visual perspective(“first person view”). Thus, embodiments disclosed herein include anAnimal-Wearable First Person View (AW-FPV) digital capture andcommunications system that enables direct observation of an animal'sdirect line of sight, visual communication back to the animal, as wellas bidirectional audio communication between the device and a remotehuman handler. Such embodiments may include: 1) a video camera systemfor the direct observation by a human of an animal's visual perspectivethat can be attached to existing animal worn hardware, objects, devices,and the like, 2) a video and auditory communications system for thedirect observation by a human of an animal's visual perspective, andbidirectional audio feedback that can be attached to existing animalworn hardware, objects, devices, and the like, and 3) a video system forthe direct observation by a human of an animal's visual perspective, andprojecting light emitting sources for visual communication back to theanimals that can be attached to existing animal worn hardware, objects,devices, and the like.

The AW-FPV system described herein may be integrated with an AW-HMD suchas described above. Alternatively, the AW-FPV system may be a separatedevice that can be used alone or in conjunction with an AW-HMD system,such as AW-HMD system 1 described above.

Some embodiments of the AW-FPV system include the following components:a ruggedized housing with simple screw mount designed to mount flushwith animal worn goggles, and one or more cameras integrated into thehousing and capable of capturing the visual perspective of the animal.The AW-FPV system (also called “AW-FPV device” herein) may furtherinclude one or more microprocessors, controllers, wireless transceivers,or the like, and memory to store software and data. The AW-FPV systemmay also have one or more of the following: speaker; microphone;infrared camera; visible-light and/or infrared light sources, thermalcamera; accelerometer; and/or magnetometer; gimbal; fold mirror(s);visible light sources. A microphone or microphones may be used toprovide ambient or active auditory signals to a remote human handler. Aspeaker or speakers enable auditory commands from a remote human handler(or a computer system) to be given to the animal or to other persons oranimals in the vicinity of the device. An outward facing light emittingsource enables display of visual cues back into the line of sight of theanimal. Further, additional sensors may also accompany the system alongwith custom software to specifically address issues related to mammalspecific or species specific visual perception, morphology, physiologyand the like. Other features of the AW-FPV system may include digitaland/or mechanical image stabilization to account for the particularmorphology and movement patterns of mammals or specific species.

FIG. 10 shows a front and bottom perspective view of an embodiment ofthe AW-FPV device. FIG. 11 shows how such an embodiment can mount to anAW-HMD designed for a canine, such as described above. In thisembodiment, the AW-FPV device 1101 is a sensor module that includes twocamera modules 1200 (such as the Leopard Imaging LI-OV9712-USB-M8),illuminating LEDs 1201 (such as the Osram SFH 4710), a microphone 1202and status LEDs 1203 for ease of use.

The FOV provided by the camera or cameras 1200, in some embodiments, maybe fixed and in others may be variable dynamically (i.e., duringoperation). Additionally, in this context the FOV may relate to verticalor horizontal orientations. Moreover, in some embodiments, the cameras1200 may also have mechanical image stabilization, such that the imagecaptured by the camera is stable in both static and dynamic scenarios.In some embodiments, the cameras 1200 may also have dynamic pan and zoomfunctionality, controlled by the associated processing capability of theAW-FPV system. In other instances, pan and zoom functionality may becontrolled by a human user through remote user interfaces presented on aremote processing device, via a wireless (or wired) network connection(similarly to as described above in connection with the AW-HMDembodiments). In other embodiments, the ability to manipulate pan andzoom, as well as camera orientation and resulting field of view, may beadjusted manually at the AW-FPV device 1101 device itself.

In some embodiments, the illuminating LEDs 1201 may be aligned to thefield of view of the cameras 1200. In the case that a remote userinterface is being used to remotely view the captured image andenvironment of the AW-FPV device 1101 through a network connection, theobserver may choose to ensure that the illuminating lights match the FOVof the camera, such that the best possible image is presented to theremote user interface. In other instances, it may be desirable to set anarrow and bright path, or a more widely distributed illuminationstream; more confined than the camera 1200 FOV, or well beyond the FOVof the camera 1200. In these instances, the user is able to make thisdynamic adjustment to the AW-FPV device 1101 through a remote userinterface, via a networked connection. In some embodiments, however, asdescribed herein, a user of the AW-FPV system may wish to manipulate theilluminating perspective of the LEDs 1201, manually. In such a case, itmay be possible for manual adjustment directly on the AW-FPV device 1101itself, to adjust the LEDs 1201 to the desired illumination angle.

Some embodiments of the AW-FPV device 1101 may contain indicators 1203both for functional and aesthetic purposes. Indicators 1203 are assumedto be LEDs herein to facilitate description, but can be any other typeof indicator. The LED indicators 1203 can provide indication of thesystem's successful operation. Alternatively or additionally, the LEDindicators 1203, may simply provide a useful yet aesthetic complement tothe overall design of the AW-FPV device 1101, controlled either by aremote user interface 103 through a networked connection 104 and/orthrough manual manipulation at the AW-FPV device 1101. The use offunctional and/or aesthetic LEDs 1203 are not mutually exclusive, nor isthe number of LEDs 1203 limited to the number shown.

The optional microphone 1202 enables the AW-FPV device 1101 to discerncardioid audio patterns, omnidirectional audio, and/or both. Moreover,based on what is desirable, some embodiments may include more than onemicrophone 1202, such that appropriate audio coverage is captured.

Some embodiments of the AW-FPV device 1101 may include protectivecoverings 1204 of the elements described herein. Protective coverings1204 allow for normal operation of the equipment they are meant toprotect, which in this embodiment may be transparent covers to LEDs 1201and cameras 1200. However, in some embodiments it may be desirable forthe protective coverings 1204 to modify the light entering the cameras1200 and/or emitted by the illuminating LEDs 1201, such that the imageis modified in a desirable way to facilitate observation through aremote user interface.

The configuration illustrated in FIG. 10 can be lightweight and compact,such that it fits flush against a portion of an animal based device,object, or mount, such as the AW-HMD described above and in FIGS. 1through 9. Such a configuration is shown in FIG. 11. Other embodimentsof the AW-FPV device 1101 may only include one camera and no otherimaging or sensing components, thus, reducing the form factor of theoverall AW-FPV device 1101. In other embodiments, the AW-FPV device 1101may include additional cameras, more or fewer LEDs, more or fewermicrophones, and/or may be of a smaller or larger size.

Some embodiments of the AW-FPV device 1101 may include outwardlyprojecting light sources, such that a directed beam of light that can bepointed in a direction at the discretion of a user through a remote userinterface through a networked connection. Moreover, through detection bythe camera(s) 1201 and use of associated processing power in the AW-FPVdevice 1101 and/or the remote device, the directed light source can beautomatically aimed toward objects of interest in the environment 102(e.g., an object that has just entered the FOV), based, for example, onuser-specified settings in the AW-FPV device 1101.

FIG. 12 illustrates a side and partial front view of the AW-FPV device1101 in accordance with the principles introduced here. In thisembodiment, the AW-FPV device 1101 includes speakers 1300 on the sideAW-FPV device 1101. For some embodiments it may be desirable to provideauditory cues to the animal wearing the AW-FPV device 1101. As such,directional placement or orientation of the speakers may be optimized totake advantage of typical mammalian auditory characteristics andmorphology, including the ability for the AW-FPV device 1101 to producebinaural sounds and/or generate sounds at a frequency discernable onlyto the animal. In some embodiments, however, it may be desirable to usea speaker 1300 to communicate broadly with the ambient environment, inwhich case the location of the speaker 1300 may not be on the side orsides of the AW-FPV device 1101, rather, it may be located, on the topof the AW-FPV device 1101, for example.

The AW-FPV device 1101, via a remote user interface for viewinginformation provided by the AW-FPV device 1101 via a networkedconnection, may also allow for dynamic manipulation of the speakers1300, based on real-time scenarios. For example, it may be desirable fora remote user to toggle between frequencies delivered by the speaker1300. Moreover, because of the AW-FPV device 1101 is able to detectenvironmental conditions through sensors and associative processingpower, the AW-FPV device 1101 may also dynamically modulate thefrequency and/or volume level delivered through the speakers 1300 basedon real-time conditions.

In the illustrated embodiment, the housing for the AW-FPV device isconstructed in two primary pieces: a primary housing 1302, and a backhousing 1301. In this embodiment the primary housing 1302 and backhousing 1301 are connected and mounted to the top portion of an animalworn goggle, such as that produced by Rex Specs, using two fasteners,each comprising a screw and a mounting plate. In other embodiments, itmay be desirable for the housing of the AW-FPV device 1101 to beconstructed in a uni-body fashion or in more than two primarycomponents.

FIG. 13 shows an example of the internal components of the AW-FPV device1101, i.e., without the housing, including cameras 1200, illuminatingLEDs 1201, microphone 1202, LED indicators 1203 and speakers 1300. Itshould be noted that this mechanical configuration of parts and allother embodiments and examples presented herein are only illustrative,not limiting. As described, the AW-FPV device 1101, may include fewer ormore of each component shown in this drawing, and therefore, the actualconfiguration and location of each component piece may be adjusted tofit the requirements of the AW-FPV system, generally. For instance, theillustrated embodiment of the AW-FPV device 1101 may be suitable for usewith the large size of goggles commercially available from Rex Specs. Inother embodiments, it may be desirable to reduce the size of the AW-FPVdevice.

The AW-FPV device may also include additional components, which are notshown, such as a power supply (e.g., battery), microprocessor, memory,wireless transceiver and antenna. In some embodiments, those additionalcomponents may be located within or integral to the housing. Forexample, the antenna can be of a form that wraps around at least part ofthe housing of the AW-FPV device. The processor, memory and wirelesstransmitter can be part of a system-on-a-chip (SoC) mounted on acustom-built printed circuit board (PCB). The battery can be a customrechargeable battery pack. In other embodiments, at least some of thoseadditional components may be external to the housing; for example, someof all of those components may be mounted on or in a vest 421 worn bythe animal 422, as shown in FIG. 4D, discussed above. These componentsmay be contained within one or more physical modules 423 that areattached to the vest and connected to the head-mounted portion of theAW-FPV device by one or more cables for providing power to thehead-mounted portion and communicating data between the head-mountedportion and the components in the module(s) 423. In other embodiments,the one or more modules may communicate with and/or provide power to thehead-mounted portion wirelessly.

FIG. 14 illustrates an embodiment of the mounting and locking mechanismsof the AW-FPV device 1101, that secures the AW-FPV device 1101 to ananimal worn device head fitting, such as that of the AW-HMD describedabove, aligning it to the point of view of the animal. In someembodiments it may be desirable to have dedicated mounting. In theembodiment shown in FIG. 14, each mounting and locking plate 1500attaches to two separate points on the AW-FPV device 1101 and existinganimal worn device. The design of the mounting and locking plates 1500shown in FIG. 14 is designed to fit the particular ribbing 1503 providedin the upper area of the goggles, such as Rex Specs goggles in theillustrated embodiment shown from the top view of a cross-sectionillustration. The AW-FPV device 1101 may be attached to the animal worndevice by putting the plates 1500 underneath the top surface ribbing ofthe animal worn device, with the flat, triangular protrusions 1502 onplates 1500 making contact with the bottom surface of the housing of theAW-FPV device 1101, and the AW-FPV device 1101, which are secured inplace by screws 1501 directly from the inside of the AW-FPV device 1101through existing animal worn device into screw holes in the mounting andlocking plates 1500.

Additional embodiments may also include snap and lock mechanisms with acantilever, whereby it may be possible to manually retract thecantilever hook, place the AW-FPV device 1101 onto the surface of whichit is to be mounted, in this case, Rex Specs goggles, and upon releasingthe cantilever hooks, the hooks would naturally lock the AW-FPV device1101 into position by locking into place by securing itself to theribbing 1503 on the top of the Rex Specs goggles. Moreover, additionalmethods for securing the AW-FPV device 1101 may also be desirable suchthat the AW-FPV device 1101, can easily secure to an existing device, orobject by use of clips. In such an embodiment, pressure mounted clipsmay be mounted to the AW-FPV device 1101, securing in a similar fashioninstead of screws 1501 in FIG. 14, although the clips may protrude fromthe AW-FPV device 1101 and allow for easy attaching and detaching fromthe top ribbing 1503 of the Rex Specs goggle.

In the illustrated embodiments, the orientation of the front housing1302, back housing 1301, mounting mechanisms, and overall design havebeen made with respect to the morphological characteristics of mammals,such that the principles do not adhere to traditional orthogonalhuman-worn devices. Moreover, the non-planar facial structure of mostmammalian species indicates that any animal worn device, specificallyrelating to devices worn on the head, will tend to have a horizontalrotation. Thus, it may be desirable for some applications of the AW-FPVdevice 1101 to counter this horizontal rotation tendency of the mountingsurface, by mounting the AW-FPV device 1101 in such a way that it canovercome the rotation, giving it a downward focused appearance. In otherembodiments, in may be desirable that, instead of orienting the AW-FPVdevice 1101 in its entirety in a downward manner, the individualcomponents making up the AW-FPV device 1101, such as the cameras 1200and the like, be mounted within the AW-FPV device 1101 with theappropriate downward angle to accommodate for the specific headorientation of the animal such that the observer through a remote userinterface 103 may mirror the viewing angle of the animal wearing theAW-FPV device 1101.

FIG. 15 illustrates an example of mounting of the AW-FPV device 1101 tocanine-adapted goggles 1600, such as those made by Rex Specs (only halfof the goggles 1600 is shown, to enable better visualization of theAW-FPV device 1101). In this embodiment, the elements illustrated inFIGS. 11 and 12 are evident and assumed. Other embodiments of the AW-FPVdevice 1101 may also be desired such that a lower profile is madepossible and/or components of the system are given further protection bybeing mounted inside of the body of the goggle itself. Morespecifically, it may be desirable for the cameras 102 to be flush withthe top portion of the goggles. As such, some embodiments may have thecameras 102 laid flat, pointing vertically, and may employ fold mirrorsto provide the camera a view of the outside world. In other embodiments,the cameras 102 may be mounted inside the goggle, providing additionalprotection from the elements. It may also be desirable that with anembodiment aimed at decreasing the size of the form factor of the AW-FPVdevice 1101, and thus more seamlessly integrating into the AW-FPV device1101, the illuminating LEDs 102 may also be decreased in height and/ormay have different mounting locations, where, in the case of using RexSpecs goggles, for example, the Illuminating LEDs 102 would be designedto fit over the molding 1601 of the goggles, covering the molding 1601in which the lens 1602 of the Rex Specs goggles is secured.

FIG. 16 depicts an example of a remote user interface 1703 for use by ahuman observer via a network connection, with respect to the use of theAW-FPV device 1101. The animal wearer of AW-FPV device 1101 mayencounter person 1702 and/or other objects of interest in a typicalrural, urban, and/or suburban environment or the like, images are ofwhich captured by the AW-FPV device 1101 and displayed on the remoteuser interface 1703. A human user of the remote user interface 1703 maychoose to engage with the captured scene in myriad ways that aredesirable to them. In one embodiment, the observer may choose to recorda sequence of events being observed by the AW-FPV device 1101 to thecloud, locally to the AW-FPV device 1101 through an internal orremovable storage device, or the remote user interface 1703, or thelike.

In certain embodiments, the remote observer of the environment beingcaptured by the AW-FPV device 1101 may choose to manually change,control or configure cameras 1200 on the AW-FPV device 1101, and/orchange, control or configure the radiance and intensity of theilluminating LEDs 1201, by using a control 1701 (for example, a button,slider, or toggle switch) displayed on the remote user interface 1703.Further, it may be desirable for the observer to modify the picturebeing displayed on the remote user interface such that the picture moreclosely matches the acuity and perception of an animal or specieswearing the AW-FPV device 1101.

The remote device that generates the remote user interface may be asecondary HMD device, worn by a human, such as an AR, MR and/or virtualreality (VR) headset, such that the wearer of the secondary HMD devicecan be virtually immersed in the environment of the animal. The remoteuser interface may be interactive, such that the wearer of the secondaryHMD device, through gesture, voice commands, or the like, can affect oradjust operation of the AW-FPV and/or the AW-HMD (e.g., to guide theanimal, adjust device settings, etc.)

The AW-FPV device 1101 may include one or more processors to performand/or control the various functions of the AW-FPV mentioned above, aswell as one or more wireless transceivers, memory and other electroniccomponents. To that extent, the block diagram of FIG. 9, discussed abovein relation to the AW-HMD, may also be representative of thearchitecture and major components of the AW-FPV device 1, according toat least one embodiment. In embodiments that include both an AW-HMD andan AW-FPV device, at least some components may be shared by the AW-HMDand an AW-FPV device.

Unless contrary to physical possibility, it is envisioned that (i) themethods/steps described herein may be performed in any sequence and/orin any combination, and that (ii) the components of respectiveembodiments may be combined in any manner.

The machine-implemented operations described above can be implemented byprogrammable circuitry programmed/configured by software and/orfirmware, or entirely by special-purpose circuitry, or by a combinationof such forms. Such special-purpose circuitry (if any) can be in theform of, for example, one or more application-specific integratedcircuits (ASICs), programmable logic devices (PLDs), field-programmablegate arrays (FPGAs), system-on-a-chip systems (SOCs), etc.

Software or firmware to implement the techniques introduced here may bestored on a machine-readable storage medium and may be executed by oneor more general-purpose or special-purpose programmable microprocessors.A “machine-readable medium”, as the term is used herein, includes anymechanism that can store information in a form accessible by a machine(a machine may be, for example, a computer, network device, cellularphone, personal digital assistant (PDA), manufacturing tool, any devicewith one or more processors, etc.). For example, a machine-accessiblemedium includes recordable/non-recordable media (e.g., read-only memory(ROM); random access memory (RAM); magnetic disk storage media; opticalstorage media; flash memory devices; etc.), etc.

The term “logic”, as used herein, means: a) special-purpose hardwiredcircuitry, such as one or more application-specific integrated circuits(ASICs), programmable logic devices (PLDs), field programmable gatearrays (FPGAs), or other similar device(s); b) programmable circuitryprogrammed with software and/or firmware, such as one or more programmedgeneral-purpose microprocessors, digital signal processors (DSPs) and/ormicrocontrollers, system-on-a-chip systems (SOCs), or other similardevice(s); or c) a combination of the forms mentioned in a) and b).

Examples of Certain Embodiments

Certain embodiments of the technology introduced herein are summarizedin the following numbered examples:

1. A head-mountable display device comprising: a head fitting designedto fit the head of an animal; and an output subsystem coupled to orintegral with the head fitting and configured to output a signal to theanimal.

2. A head-mountable display device as recited in example 1, wherein theoutput subsystem comprises an optical module configured to project animage into an eye of the animal.

3. A head-mountable display device as recited in any of example 1 orexample 2, wherein the optical module comprises a display element thatis at least partially transparent, on which to overlay augmented realityimages and/or mixed reality images on the animal's view of the realworld.

4. A head-mountable display device as recited in any of examples 1through 3, wherein the optical module is configured to display imagesusing shapes and colors optimized for visual perception capabilities ofa particular animal species.

5. A head-mountable display device as recited in any of examples 1through 4, wherein the optical module is configurable to generate imagesconforming to a plurality of different visual acuity and visualperception requirements for a plurality of different animal species.

6. A head-mountable display device as recited in any of examples 1through 5, wherein the output subsystem comprises an audio outputdevice.

7. A head-mountable display device as recited in any of examples 1through 6, wherein the output subsystem comprises a tactile outputdevice.

8. A head-mountable display device as recited in any of examples 1through 7, wherein the output subsystem comprises an olfactory outputdevice.

9. A head-mountable display device as recited in any of examples 1through 8, wherein the output subsystem comprises at least two from thelist consisting of: an optical module configured to project an imageinto an eye of the animal an audio output device; a tactile outputdevice; or an olfactory output device.

10. A head-mountable display device as recited in any of examples 1through 9, further comprising a camera arranged to perform eye trackingof an eye of the animal.

11. A head-mountable display device as recited in any of examples 1through 10, further comprising a wireless receiver to receive a wirelesscommunication signal via a wireless communication link in response to auser input from a remote user, the wireless communication signals forcontrolling a function of the head-mountable display device.

12. A head-mountable display device as recited in any of examples 1through 11, wherein the output system includes an output devicecontrollable by the remote user via the wireless communication signal,to provide the signal to the animal.

13. A head-mountable display device as recited in any of examples 1through 12, wherein the signal to the animal is a visual signal.

14. A head-mountable display device as recited in any of examples 1through 13, wherein the signal to the animal is an auditory signal.

15. A head-mountable display device as recited in any of examples 1through 14, wherein the signal to the animal is a tactile signal.

16. A head-mountable display device as recited in any of examples 1through 15, wherein the signal to the animal is an olfactory signal.

17. A head-mountable display device as recited in any of examples 1through 16, further comprising: a camera arranged to acquire viewpointimages corresponding to a visual perspective of the animal; and awireless transmitter configured to transmit image data representing theviewpoint images to a remote device.

18. A head-mountable display device comprising: a head fitting designedto fit the head of an animal; an optical module configured to project animage into an eye of the animal, wherein the optical module includes adisplay element that is at least partially transparent, on which tooverlay augmented reality images and/or mixed reality images on theanimal's view of the real world; and at least two from the listconsisting of: an audio output device, a tactile output device and anolfactory output device.

19. A head-mountable display device as recited in example 18, furthercomprising a wireless receiver to receive a wireless communicationsignal via a wireless communication link in response to a user inputfrom a remote user, the wireless communication signals for controlling afunction of the head-mountable display device.

20. A head-mountable display device as recited in example 18 or example19, wherein the function is one of a visual signal to the animal, anauditory signal to the animal, a tactile signal to the animal or anolfactory signal to the animal.

21. A head-mountable display device as recited in any of examples 18through 20, further comprising: a camera arranged to acquire viewpointimages corresponding to a visual perspective of the animal; and awireless transmitter configured to transmit image data representing theviewpoint images to a remote device.

22. A method of operation of a head-mountable display device designed tobe worn by an animal, the method comprising: projecting a plurality ofimages into an eye of the animal from a display element mounted to thehead of the animal; receiving, via a wireless communication link, acontrol signal from a remote user; and generating an output signal tothe animal based on the control signal.

23. A method as recited in example 22, wherein the output signal isincluded in the plurality of images.

24. A method as recited in example 22 or example 23, wherein the outputsignal is a non-visual signal.

25. A method as recited in any of examples 22 through 24, wherein theoutput signal is an audible signal.

26. A method as recited in any of examples 22 through 25, wherein theoutput signal is a tactile signal.

27. A method as recited in any of examples 22 through 26, wherein theoutput signal is an olfactory signal.

28. A method as recited in any of examples 22 through 27, furthercomprising: acquiring viewpoint images corresponding to a visualperspective of the animal; and transmitting image data representing theviewpoint images from the device to a remote device.

29. A sensor module comprising: a housing; at least a portion of afastener, the fastener configured to removably attach the housing to ahead fitting designed to fit the head of an animal; and a first cameraat least partially contained within the housing and positioned toacquire image data of a real world environment of the animal when inoperation while the sensor module is attached to the head fitting andthe head fitting is worn by the animal.

30. A sensor module as recited in example 29, wherein the head fittingcomprises goggles.

31. A sensor module as recited in example 29 or 30, further comprising asecond camera at least partially contained within the housing andpositioned to acquire image data of the real world environment of theanimal when in operation while the sensor module is attached to the headfitting and the head fitting is worn by the animal.

32. A sensor module as recited in any of examples 29 through 31, furthercomprising a microphone at least partially contained within the housingand positioned to acquire audio data of the real world environment ofthe animal when in operation while the sensor module is attached to thehead fitting and the head fitting is worn by the animal

33. A sensor module as recited in any of examples 29 through 32, whereinthe head fitting comprises goggles, the sensor module further comprisinga plurality of light emission elements at least partially containedwithin the housing and positioned to illuminate a field of view of theanimal when in operation while the sensor module is attached to thegoggles and the goggles is worn by the animal.

34. A sensor module as recited in any of examples 29 through 33, furthercomprising an audio speaker.

35. A sensor module as recited in any of examples 29 through 34, furthercomprising a second camera at least partially contained within thehousing and positioned to acquire image data of the real worldenvironment of the animal when in operation while the sensor module isattached to the goggles and the goggles is worn by the animal; aplurality of light emission elements at least partially contained withinthe housing and positioned to illuminate a field of view of the animalwhen in operation while the sensor module is attached to the goggles andthe goggles is worn by the animal; a microphone at least partiallycontained within the housing and positioned to acquire audio data of thereal world environment of the animal when in operation while the sensormodule is attached to the goggles and the goggles is worn by the animal;and an audio speaker.

36. A head-mountable first person viewing system comprising: a headfitting designed to fit the head of an animal; a first camera coupled tothe head fitting and positioned to acquire image data of a real worldenvironment of the animal when the head-mountable first person viewingsystem is worn by the animal; and a wireless transmitter configured totransmit the image data in real-time, for delivery to a remote deviceconfigured to display images of the real world environment of theanimal, based on the image data.

37. A head-mountable first person viewing system as recited in example36, wherein the head fitting comprises goggles.

38. A head-mountable first person viewing system as recited in example36 or 37, wherein the camera is at least partially contained within asensor module that is removably attached to the goggles.

39. A head-mountable first person viewing system as recited in any ofexamples 36 through 38 wherein the sensor module is removably attachedto a top surface of the goggles.

40. A head-mountable first person viewing system as recited in any ofexamples 36 through 39, wherein the top surface of the goggles comprisesa webbing, the webbing including spaced apart segments of solidmaterial, and wherein the sensor module is removably attached to thegoggles by fasteners, at least portions of which pass through air gapsbetween the segments of solid material in the webbing.

41. A head-mountable first person viewing system as recited in any ofexamples 36 through 40, further comprising a second camera positioned toacquire image data of the real world environment of the animal when inoperation while the head fitting is worn by the animal; and a pluralityof light emission elements positioned to illuminate a field of view ofthe animal when in operation while the head fitting is worn by theanimal.

42. A head-mountable first person viewing system as recited in any ofexamples 36 through 41, further comprising a housing coupled to the headfitting, the housing at least partially containing the first and secondcameras and the plurality of light emission elements.

43. A head-mountable first person viewing system as recited in any ofexamples 36 through 42, further comprising a microphone at leastpartially contained within the housing and positioned to acquire audiodata of the real world environment of the animal when in operation whilethe head fitting is worn by the animal.

44. A head-mountable first person viewing system as recited in any ofexamples 36 through 43, further comprising an audio speaker.

45. A head-mountable input/output (I/O) system comprising: a headfitting designed to fit the head of an animal; an optical outputsubsystem coupled to or integral with the head fitting and configured tooutput images to the animal; a plurality of cameras, each at leastpartially contained within the housing and positioned to acquire imagedata of a real world environment of the animal when in operation whilethe head-mountable I/O system is worn by the animal; and a wirelesstransmitter configured to transmit the image data in real-time, fordelivery to a remote device configured to display images of the realworld environment of the animal, based on the image data.

46. A head-mountable I/O system as recited in example 45, wherein thehead fitting comprises goggles.

47. A head-mountable I/O system as recited in example 45 or 46, whereinthe camera is at least partially contained within a sensor module thatis removably attached to the goggles.

48. A head-mountable I/O system as recited in any of examples 45 through47, wherein the sensor module is removably attached to a top surface ofthe goggles.

49. A head-mountable I/O system as recited in any of examples 45 through48, wherein the top surface of the goggles comprises a webbing, thewebbing including spaced apart segments of solid material, and whereinthe sensor module is removably attached to the goggles by fasteners, atleast portions of which pass through air gaps between the segments ofsolid material in the webbing.

50. A head-mountable I/O system as recited in any of examples 45 through49, further comprising a microphone at least partially contained withinthe housing and positioned to acquire audio data of the real worldenvironment of the animal when in operation while the head fitting isworn by the animal; and a plurality of light emission elements at leastpartially contained within the housing and positioned to illuminate afield of view of the animal when in operation while the head fitting isworn by the animal.

51. A head-mountable I/O system as recited in any of examples 45 through50.

Any or all of the features and functions described above can be combinedwith each other, except to the extent it may be otherwise stated aboveor to the extent that any such embodiments may be incompatible by virtueof their function or structure, as will be apparent to persons ofordinary skill in the art. Unless contrary to physical possibility, itis envisioned that (i) the methods/steps described herein may beperformed in any sequence and/or in any combination, and that (ii) thecomponents of respective embodiments may be combined in any manner.

Although the subject matter has been described in language specific tostructural features and/or acts, it is to be understood that the subjectmatter defined in the appended claims is not necessarily limited to thespecific features or acts described above. Rather, the specific featuresand acts described above are disclosed as examples of implementing theclaims and other equivalent features and acts are intended to be withinthe scope of the claims.

What is claimed is:
 1. A sensor module comprising: a housing; at least aportion of a fastener, the fastener configured to removably attach thehousing to a head fitting designed to fit the head of an animal; and afirst camera at least partially contained within the housing andpositioned to acquire image data of a real world environment of theanimal when in operation while the sensor module is attached to the headfitting and the head fitting is worn by the animal.
 2. A sensor moduleas recited in claim 1, wherein the head fitting comprises goggles.
 3. Asensor module as recited in claim 1, further comprising: a second cameraat least partially contained within the housing and positioned toacquire image data of the real world environment of the animal when inoperation while the sensor module is attached to the head fitting andthe head fitting is worn by the animal.
 4. A sensor module as recited inclaim 1, further comprising: a microphone at least partially containedwithin the housing and positioned to acquire audio data of the realworld environment of the animal when in operation while the sensormodule is attached to the head fitting and the head fitting is worn bythe animal.
 5. A sensor module as recited in claim 1, wherein the headfitting comprises goggles, the sensor module further comprising: aplurality of light emission elements at least partially contained withinthe housing and positioned to illuminate a field of view of the animalwhen in operation while the sensor module is attached to the goggles andthe goggles is worn by the animal.
 6. A sensor module as recited inclaim 1, further comprising an audio speaker.
 7. A sensor module asrecited in claim 1, further comprising: a second camera at leastpartially contained within the housing and positioned to acquire imagedata of the real world environment of the animal when in operation whilethe sensor module is attached to the goggles and the goggles is worn bythe animal; a plurality of light emission elements at least partiallycontained within the housing and positioned to illuminate a field ofview of the animal when in operation while the sensor module is attachedto the goggles and the goggles is worn by the animal; a microphone atleast partially contained within the housing and positioned to acquireaudio data of the real world environment of the animal when in operationwhile the sensor module is attached to the goggles and the goggles isworn by the animal; and an audio speaker.
 8. A head-mountable firstperson viewing system comprising: a head fitting designed to fit thehead of an animal; a first camera coupled to the head fitting andpositioned to acquire image data of a real world environment of theanimal when the head-mountable first person viewing system is worn bythe animal; and a wireless transmitter configured to transmit the imagedata in real-time, for delivery to a remote device configured to displayimages of the real world environment of the animal, based on the imagedata.
 9. A head-mountable first person viewing system as recited inclaim 8, wherein the head fitting comprises goggles.
 10. Ahead-mountable first person viewing system as recited in claim 9,wherein the camera is at least partially contained within a sensormodule that is removably attached to the goggles.
 11. A head-mountablefirst person viewing system as recited in claim 10, wherein the sensormodule is removably attached to a top surface of the goggles.
 12. Ahead-mountable first person viewing system as recited in claim 11,wherein the top surface of the goggles comprises a webbing, the webbingincluding spaced apart segments of solid material, and wherein thesensor module is removably attached to the goggles by fasteners, atleast portions of which pass through air gaps between the segments ofsolid material in the webbing.
 13. A head-mountable first person viewingsystem as recited in claim 8, further comprising: a second camerapositioned to acquire image data of the real world environment of theanimal when in operation while the head fitting is worn by the animal;and a plurality of light emission elements positioned to illuminate afield of view of the animal when in operation while the head fitting isworn by the animal.
 14. A head-mountable first person viewing system asrecited in claim 13, further comprising a housing coupled to the headfitting, the housing at least partially containing the first and secondcameras and the plurality of light emission elements.
 15. Ahead-mountable first person viewing system as recited in claim 14,further comprising: a microphone at least partially contained within thehousing and positioned to acquire audio data of the real worldenvironment of the animal when in operation while the head fitting isworn by the animal.
 16. A head-mountable first person viewing system asrecited in claim 15, further comprising an audio speaker.
 17. Ahead-mountable input/output (I/O) system comprising: a head fittingdesigned to fit the head of an animal; an optical output subsystemcoupled to or integral with the head fitting and configured to outputimages to the animal; a plurality of cameras, each at least partiallycontained within the housing and positioned to acquire image data of areal world environment of the animal when in operation while thehead-mountable I/O system is worn by the animal; and a wirelesstransmitter configured to transmit the image data in real-time, fordelivery to a remote device configured to display images of the realworld environment of the animal, based on the image data.
 18. Ahead-mountable I/O system as recited in claim 17, wherein the headfitting comprises goggles.
 19. A head-mountable I/O system as recited inclaim 18, wherein the camera is at least partially contained within asensor module that is removably attached to the goggles.
 20. Ahead-mountable I/O system as recited in claim 19, wherein the sensormodule is removably attached to a top surface of the goggles.
 21. Ahead-mountable I/O system as recited in claim 20, wherein the topsurface of the goggles comprises a webbing, the webbing including spacedapart segments of solid material, and wherein the sensor module isremovably attached to the goggles by fasteners, at least portions ofwhich pass through air gaps between the segments of solid material inthe webbing.
 22. A head-mountable I/O system as recited in claim 17,further comprising: a microphone at least partially contained within thehousing and positioned to acquire audio data of the real worldenvironment of the animal when in operation while the head fitting isworn by the animal; and a plurality of light emission elements at leastpartially contained within the housing and positioned to illuminate afield of view of the animal when in operation while the head fitting isworn by the animal.
 23. A head-mountable I/O system as recited in claim22, further comprising an audio speaker.